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ABSTRACT 



Taking a post-modern approach, this paper addresses the 
relevance of measurement and evaluation of libraries and information 
services. The following topics are discussed: (1) past and present approaches 

to performance measurement, including origins of measurement and landmarks of 
the past; (2) a conceptual framework for evaluation, including a typology of 
the performance literature describing four models that organizations tend to 
use to measure their effectiveness (i.e., goal attainment, systems 
resource/external systems, internal processes/internal systems, and multiple 
constituency/constituency satisfaction models) , and a proposed new model of 
organizational effectiveness (i.e., a focus/value/purpose matrix); and (3) 
factors affecting the use of systems of evaluation, including library use of 
past research. It is concluded that there are three critical factors 
influencing the positive outcome of performance measures in libraries: 
measurement is a political activity; the multidimensional nature of 
performance measurement; and rewards and incentives. A chronological list of 
performance measurement literature is appended. (Contains 21 references.) 
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INTRODUCTION 



How can you determine the best way of measuring your library's performance? Which measures are 
relevant for you? Measure for measure, which measures will make your organisation more effective and 
responsive, and relevant to your client's needs? 

As a profession we have been grappling with this question for a quarter of a century. Responses have 
focused on the semantic, the practical, the theoretical and the dogmatic. But in recent years they have 
focused mainly on the practical. Recent articles describe and explore new ways of evaluating services, 
new services to be evaluated, new issues to be considered. Terms like TQM, IS09000, Servqual and gap 
reduction are thrown around. What do they all mean? How do they relate to each other? Few writers 
have questioned the fundamental value and relevance of the process of measurement itself 

It is in fact a difficult question, because it is self referential. If we could answer yes, and simply get on 
with ascertaining the best ways in which to measure and evaluate what we do, that would imply that we 
know what an effective library is, and can measure it. We would therefore be finding it easier to define 
and adopt library performance measures, and they would be telling us yes, measuring performance leads 
to better performance and therefore more effective and relevant libraries. The fact we are here debating 
these issues suggests that we have not yet reached that point. In fact, the question we must ask is: are we 
really any closer to knowing whether performance measures lead to library effectiveness than when the 
debate began twenty-five to thirty years ago? This is the approach I have taken in this paper, and I hope 
that it will help you to find an approach to performance measurement which will help you manage your 
library better, and use measurement to enable you to deliver a better service. 



Oo 

O 



I have taken what is essentially a post-modem approach, which analyses the 'discourse' of evaluation, 
and deconstmcts the certainties of the past. By 'discourse' I mean the language, the definitions, the ways 
of expressing the concepts of evaluation that we have used. In other words, the measures we use tell us, 
presumably, what it is we value in libraries. So that in analysing the 'discourse' and examining the 
premises on which we have built our notions of evaluation and measurement, we have an opportunity to 
re-examine those values. We are thus empowered to select and choose a system of performance 
measurement that suits the individual organisation and best reflects its fundamental purpose and its 
environment. Because, a library is a social constmct, devised for a range of societal purposes and to 
meet a number of needs; library effectiveness is therefore dependent upon the insubstantial and fluid 
concept 'library', a concept which has become even more insubstantial in the age of the 'virtual library'. 
The attempt to discuss the notion of performance measurement in libraries is therefore both fhistrated 
because of its complete dependence on such relativistic foundations, and necessary, because in the midst 
of such uncertainty we must attempt to create some meaning. The discourse of performance 
measurement is one way to establish that meaning. 




Post-modernism and libraries 

A post-modem analysis finds that words, symbols, and signs are increasingly divorced from direct 
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real-world experience. For example, the world of catalogues, call numbers, collection management and 
bibliographic control may have no meaning in the age of the 'virtual library' with its electronic journals 
delivered on demand, and its 'virtual collection' of electronic documents, stored in another continent and 
accessible on a "just-in-time", rather than a "just- in-case" basis. Post-modernism challenges the 
knowledge of the past by challenging the assumptions of the scientific rational method, and by rejecting 
the grand all-encompassing theory, such as, theories of universal bibliographic access and universal 
bibliographic control. Postmodernism, rather than seek to determine models which will serve for a 
universally recognised paradigm of 'library' focuses on local and transitory meanings, which serve for a 
time, and then are discarded when they no longer have relevance. It challenges definitions, and social 
constructs, and asks ’’what do we mean by 'a library'”? and ”do you mean by 'library' what I mean by 
'library’?” and if you don't then how can measures that determine the effectiveness of your library be of 
any value to me trying to measure the effectiveness of my library? 

Ron Day would go so far as to say that library science as a discipline has been confounded from the 
outset by the fact that its very domain of study "information" is intricately linked with our concept of the 
nature of science, of the certainties of empiricism, and structure of recorded knowledge. (Day, 1996). 
Thus, the library, with its authority, structure and systems for organising knowledge might be seen as the 
scientific, modernist, certainty of the past. The Internet, by contrast can be viewed as a truly 
post-modem environment. In fact it has been described as 'the working model of post-modernism' as 
proposed by Lyotard (Hubbard, 1995: 448). Eschewing the certainties of print publication, the authority 
of peer-reviewed journals, and the certainties of empirical research, it mixes the scholarly and scientific 
with the egregiously spurious. Without the imposed orthodoxy of the world of scholarly publishing and 
libraries it reduces everything to everything to a common denominator in which the surfer is king, and 
meaning is imposed by stmctures outside its own boundaries- by the constructs we bring to it.. In this 
context we are forced to redefine the nature and purpose of libraries, and consequently the framework of 
evaluation and measurement we will bring to them. 

The paper therefore addresses the central question of the relevance of measurement and evaluation in 
libraries and information services by : 

• examining past and present approaches to performance measurement in libraries 

• putting past and present practice into a conceptual framework 

• defining effectiveness as a multi-dimensional construct 

• identifying the factors necessary for the effective application of performance measurement in 
library and information services 

PAST AND PRESENT APPROACHES TO PERFORMANCE MEASUREMENT 



Origins of measurement in libraries 

Performance measurement or evaluation has historically been used by library and information services in 
two ways. From the Great Library at Alexandria up until the present day, libraries have judged 
themselves and each other in terms of their collections and their curators. The pride taken in the early 
years of this century in the size and quality of the collection, often focused on the number of rare and 
valuable items it contained, and the distinction of the staff as scholars and luminaries in their own right, 
has dominated much library history. This view was challenged by the documentalists of the nineteen 
thirties and forties, applying the mathematics of stochastic processes to the circulation of books and 
information in library systems. They began to shift the focus from the collection to the processes going 
on within the collection and the interaction between the collection and its users. The science of 
bibliometrics and the understanding it brought of the library as part of the cycle of the creation, 
organisation and dissemination oftoowledge, changed the concept of the library as a self-contained 
entity to a dynamic system in constant interaction with its environment. 



As this understanding developed, and as the library became a more sophisticated operation, standing 
apart from the community of scholars or readers which it served, it became recognised as a social 
institution rather than a collection of materials. Principles of management, and principles of evaluation 
developing in the management literature, were now perceived to be applicable to libraries. Performance 
measurement in library and information services has since kept pace with the range of methodologies 
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used in the profit, and not-for-profit sectors — Strategic Planning, Systems Analysis, Management 
Information Systems, MBO, TQM, BPR, IS09000 and its off-shoots, and the Baldridge Award. We 
have flirted them all and engaged seriously with some. 

More recent developments reflect the role of government, and the impact of developing principles of 
'accountability' in government on libraries, a large proportion of which are owned by national, state, or 
local government or government agencies. The New Zealand government, for example, currently 
declares its social and financial goals as Key Result Areas, and Strategic Result Areas and reports 
annually on the extent to which these are achieved. It has become an international leader in the 
development of methodologies for measuring the effectiveness of government, and on ways to define the 
services it purchases on behalf of the community from provider agencies, such as universities and 
libraries. (Boston, 1996). We will need to see where such developments fit into our analysis. 

What does emerge from these various approaches to measurement and evaluation are a set of 
fundamental premises about measurement. The methodology is of less significance than the fact that 
some organisational endeavour is given over to the measurement of performance. Two key principles 
then can be seen to be: 

• that the organisation seeks feedback on the impact of its main endeavours, i.e. that the feedback 
loop is closed 

• that the measurement of performance is tied to decision-making and resource allocation. 

Within these parameters there is a large element of choice, and an organisation may focus its 
measurement on a range of activities. 

But this still begs the question: What is effectiveness? How do we measure it? If we do, do things 
improve? The debate in our profession, I would suggest, has focused too much on semantics and 
process, and too little on outcomes. The questions we should therefore be asking are: 

• what performance measures are libraries and information services using? 

• what are these measures actually measuring ? 

• can libraries and information services demonstrate changes and improvements in procedure and 
resource allocation as a result of using these measures? 

Landmarks of the past 

One way of answering these fundamental questions may lie in a more clearly defined conceptual 
framework for the process of output measurement, the measurement of outcomes, and the evaluation of 
services. 

In order to develop such a framework we need to go back to some of the landmarks of the past 30 years, 
30 years of an extensive literature to which we have already briefly referred. Here are sonie notable 
pieces of work, glimmerings of real knowledge in the research literature, in monographs, in conference 
proceedings, in handbooks and manuals - a "massif with some named peaks - research, methods, and 
insights, which have added to the debate, changed its direction, made some significant impact on 
thinking about the issue. (See Appendix). This is not by any means a definitive list, even of the 
acknowledged seminal papers, or 'landmarks'. It is used simply to point out some trends. But most 
names of note are included, especially those of Kant or, and Orr, two key figures who made significant 
conceptual breakthroughs that are only beginning to be fully explored. And some are briefly mentioned 
who are far more prolific than this list suggests, especially major figures such as Buckland and McClure. 

Of note in this list is a progression from the early documentalists, using statistical methods to analyse 
patterns of use in library collections, developing concepts of measurement, early applications of the 
'systems' approach, identification of need for 'objective' measures, linkage of planning (and goals and 
objectives) with measurement, the concept of 'goodness' and whether statistics can measure goodness or 
not, the emergence of 'effectiveness', MIS and DSS, and finally the introduction of concepts of quality, 
IS09000, the distinction between satisfaction and service quality, and the definition of the attributes of 
service quality. 
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During this period (the past 25-30 years) we have learned a lot about library performance and 
measurement. We have learned to distinguish between inputs and outputs, to understand the difference 
between measuring processes, and measuring products. We learned about the need for feedback. We 
have learned also that outputs may not equate with outcomes, and that to find out how well we are doing 
we really need to determine what impact library and information services have on the immediate 
community we work in and on society at large. And then we leaned that these outcomes are more 
difficult concept s to measure than inputs or outputs or even processes. They are fuzzy, contradictory 
and often unquantifiable. Our state of knowledge might be simply expressed thus; 



inputs 

resources 

(human/ 
financial etc) 



Evaluation 

feedback 

processes outputs 

performance customers 

measures service quality 



outcomes 

?? 

satisfaction 



Figure 1. The three aspects of evaluation 
A CONCEPTUAL FRAMEWORK FOR EVALUATION 
A typology of the performance literature 

A more sophisticated typology which has been used in our discipline provides some insights into why 
there is such a range of evaluative methodologies, and how they relate to each other. Advantages and 
disadvantages of each can then be assessed and choices made as to which method is the more 
appropriate. 

This typology, as it is used in Library and Information Science, and other educational and 
quasi-govemmental industries is generally attributed to Kim Cameron. Although it is usefully applied to 
the not-for-profit sector, since it provides other ways than profit for measuring organisational 
effectiveness, it originated in the for-profit sector and is found in a number of management textbooks, 
with various attributions. It was used by Van House and Childers (and subsequently by Calvert and 
Cullen) in a series of studies which now amount to a substantial body of replicable research on 
dimensions of library effectiveness. 

In Cameron's typology (Cameron, 1986) there are four models which describe the ways in which 
organisations tend to measure their 'effectiveness'. They are referred to as: 

• the goal attainment model 

• the systems resource (or external systems) model 

• the internal processes, or internal systems model 

• the multiple constituency, or constituency satisfaction model. 

In the goal attainment model the organisation assesses its effectiveness in terms of the extent to which 
it achieves its goals and objectives. These may be focused on acquiring resources from the environment, 
increasing outputs, or establishing new services. Reporting and measurement, and resource allocation are 
concentrated on the extent to which these goals are achieved. Goals may also be focused on achieving 
pre-determined standards or bench-marks. Heavy reliance on output measures usually indicate that much 
of the organisation's endeavours are focused on increasing outputs, and therefore on goal achievement. 

In the systems resource (or external systems) model the organisation measures its effectiveness in 
terms of its ability to gain resources from its environment. These are input measures. Traditionally, as 
we observed above, libraries have concentrated on, have measured themselves and reported in terms of: 
the size of the budget, number of staff, number of qualified staff, staff publications, the size of the 
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collection, strengths of the collections, etc.. Successful libraries have been considered to be those which 
secured more extensive resources from the environment, and translated them into these ostensibly 
desirable inputs. Library statistics, and benchmarks of minimum staffing levels for a particular 
population, building standards and area per capita for users, staff qualifications, collection size etc., 
reflect this model. 

In the internal processes, or internal systems model, the emphasis is on the organisation's internal 
communications systems, and the efficiency with which it converts inputs into outputs. A feedback loop 
is an important component of this model. Measures which report ratios of transactions per staff member, 
number of items added to stock per staff member, and early work on the use of library statistics for 
decision-making focus on this model. TQM and the use of the quality standards (ISO 9000/9001, etc.) , 
because the focus is as much on internal as external transactions, can also be identified with this model. 

In the multiple constituency, or constituency satisfaction model, the organisation looks outward to its 
different constituencies or stakeholder groups and measures its effectiveness in terms of the extent to 
which the needs of these different constituencies are met. The competing demands of these various 
groups must then be managed. Goals and objectives, and all other ways of measuring performance are 
constrained by the need to demonstrate to the more powerful constituencies the extent to which their 
desires are met. This is essentially a marketing model, and one that recognises the need for marketing to 
the internal as well as the external customer. 

Organisational effectiveness can therefore be seen to be a mental construct of many dimensions. 
Although organisations may not identify or articulate the model they have chosen, their choice can be 
inferred from their behaviour. Organisations may of course employ one or more of these models, in that 
the measures they select for use, or the way in which they report reflect more than one of the models 
outlined. The corollary is that as some of these models are mutually contradictory, effectiveness 
measured according to the dimensions of one model may not be compatible with effectiveness as 
measured according to the dimensions of another. 

We can now map the range of methods of performance measurement used in the examples in our 
'landscape' onto this typology. It would be preferable at this stage not to use terms like 'measurement' 
and 'effectiveness' because these are semantically compromised by their previous usage. The term 
'evaluation' is used for the typology, because it is sufficiently broad to encompass all the models that 
have been used. Figure 2 represents an initial and tentative mapping. It is not our final model. 

Evaluation 
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Goal attainment model: 



Systems resource model: 



Internal systems model: 



Multiple constituencies model: 



goals and objectives 
bench-marks 
standards 
output measures 
citizen's charter 

Input measures 
library statistics 
bench-marks 
standards 

Management Information Systems 
Decision Support Systems 
TQM- Total Quality Management 
IS09000/9001 etc. 

Service quality 
Customer satisfaction 
Total Quality Management 
IS09000/9001 etc. 
gap reduction 
marketing 



Figure 2. Systems of evaluation used in LIS mapped on to Cameron's typology 

There are some caveats relating to the use of this typology which must be mentioned here. The four 
models do not deal very adequately with the notion of outcomes (or impacts) , as distinguished from 
outputs. This is problematic for me in that the notion of outcomes has dominated much of the New 
Zealand government' s thinking in the past decade - a cost/benefit approach, or an attempt to measure the 
benefits of social service agencies in terms of social outcomes that is still in its infancy. Although if the 
government could precisely define the social benefits it desired, and was willing to fund these, it is 
possible that ways could be identified to deliver and measure them. 

Cameron's typology also does not deal well with the concept of leadership, which I will argue may be 
essential to the implementation, if not the conceptualisation of measurement. It should also be noted that 
there are some recent evaluative paradigms which straddle a number of the models here, and which bring 
a more sophisticated multi-focused approach to performance measurement, being both internally and 
externally focused. The Baldridge Award and its framework of criteria would be one such. It does not 
yet appear to have been adopted by any libraries or information services. 

As we move towards building a new conceptual framework that will accommodate the various 
endeavours of measurement of the past, and the Cameron typology, it is salutary to look one other 
model. Hemon and Altman, in Service Quality in Libraries (Hemon, 1996) distinguish 5 dimensions of 
evaluation: extensiveness, effectiveness, efficiency, costing, and quality. These can also be mapped on to 
the four models of effectiveness that we have already looked at: 
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Effectiveness - > 

Efficiency - - 



Fit with "Models of effectiveness" 

Internal processes 
External systems 
Goal attainment 

Goal attainment 
Internal processes 

Internal processes 
External processes 

Multiple constituencies 
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Figure 3. Cameron's typology mapped onto Hernon and Altman's model of evaluation 

In this paradigm, extensiveness, e.g. the 'extent' or 'amount' of inputs, of users, of time taken to process 
materials, or of services provided, is a quantitative not a qualitative measure. It relates to both the 
internal processes, or systems model, and to the external resources model, since it covers both the 
efficiency with which an organisation can convert inputs (staff, funding etc.) into outputs (reference 
enquiries dealt with, etc.) , and it also measures the organisation's ability to secure more such resources 
from the environment. If an organisation chooses to focus its planning activities on measurable 
objectives , it will be focusing its evaluation around the 'extensiveness' dimensions, rather than choosing 
to focus on other dimensions. 

Hemon and Altman focus on quality - the extent to which an organisation meets the needs of its primary 
customers as defined by a range of service attributes - as the most significant dimension of evaluation to 
which libraries should currently direct their energies. However, the Hemon/Altman typology, like 
Cameron's, reveals that an evaluative methodology which reflects only one dimension will ignore many 
other dimensions of effectiveness, or evaluation, which must betaken into account to give a true picture 
of how a library or any other service organisation is performing. This 'fore-shortened', uni -dimensional 
view may be a factor in the reluctance of libraries to whole-heartedly endorse measures proposed to date. 
An instinctive desire for a more 'holistic' approach to measurement that captures something closer to a 
truer picture of that complex social construct, the library. 

A proposed new model of organisational effectiveness: a Focus/value/purpose matrix The real value 
of Cameron's work, and that of Van House and Childers, is to demonstrate the multi-dimensional nature 
of performance measurement. Studies carried out by Childers and Van House investigating dimensions 
of effectiveness in public libraries, further explored by Calvert and Cullen in public and academic 
libraries, and by McDonald and Micikas in university and college libraries, all tend to produce the same 
broad range of results — that library performance can be shown to have a range of about 12 dimensions, 
covering management procedures, technical processes, physical plant, information/customer services etc. 
each of which is important to the whole picture, and each of which fits within one or other of the models 
of effectiveness outlined by Cameron f . 

The four models, however, can also be seen as an expression of a number of different poles or axes. One 
dimension, representing a set of organisational attributes (such as 'customer focused', 'client-centred' 
or conversely 'unresponsive') is clearly the intemal/extemal focus of the organisation. This axis is an 
expression of the degree to which the organisation focuses on and interacts with its environment. An 
organisation with an internal focus will look at its internal processes, and measure its efficiency in 
converting inputs to outputs, but will pay less attention to defining these and to their relevance to its 
ultimate purpose. 

This, then, is our first axis. 

Internal External 

focus focus 



Diagram 1 

A second axis reflects the value system of the organisation. The attributes on this axis reflect the extent 
to which the organisation places value on its inputs (i.e. its size, and the extent of the resources it 
attracts), or its outputs (the services it provides) to a range of customers. Despite our profession's focus 
on the measurement of outputs for the past fifteen years, many organisations continue to report inputs 
and clearly place great store by them. Even organisations striving to be more customer focused are still 
reporting and valuing inputs, suggesting that these two axes measure different sets of organisational 
attributes. 



UC 



8 



10/28/99 1:38 PM 



Cullen Paper 



http://educate.lib.chalmers.se/IATUL/proceedcontents/pretpap/cullen.htnil 



Values 




outputs 



Diagram 2 

However, organisations with a strong sense of their value in terms of outputs, and a strong external focus 
are likely to be highly sensitive to political issues in the environment, and to favour models of 
effectiveness which reflect their multiple constituencies. They are likely to a adopt a marketing model. 

We commented earlier on some organisational attributes that the Cameron models deal with less 
effectively. These are attributes such as purpose and leadership. We can add a third axis therefore which 
measures the extent of the organisation's resolution. Resolution is here defined as that aspect of 
organisational culture which reflects organisational unity, a sense of common purpose and movement 
towards that purpose. It is also a measure of the strength of leadership within the organisation, leadership 
that encourages change, and that shifts resources to defined goals. Organisations weak in resolution will 
tend to maintain the status quo and resist change. Their performance measurement is likely to be focused 
on existing activities and outputs. Feedback loops may exist but the information they provide will not be 
the major input in decision-making and resource allocation. Strong leadership, and a change in 
organisational culture is needed to re-orientate such an organisation to new goals. A third axis, relating 
to the strength of organisational purpose, is therefore added to our matrix which now represents a 
tri-axial figure which may be labelled a Values/focus/purpose matrix. This third axis is critical to the 
successful implementation of performance measurement. 
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The distinction between the three axes is important. An organisation which is strong on resolve and has 
strong leadership might still not have a very strong customer focus. It might focus on its inputs, its 
collections, its buildings and on gaining yet more resources of this kind, while neglecting to examine 
and increase its outputs. Conversely, an organisation with a strong customer focus may fail to pay 
attention to, and fail to succeed in gaining the resources which will enable it to meet its customers' 
needs. A third organisation might value its outputs, report on all its transactions, e.g. circulation 
statistics, reference enquiries, interloans, on-line searches, etc. and set goals to increase these each year, 
and yet have little notion of whether these transactions are meeting the needs of is primary client groups, 
or indeed have defined its primary clients or customers. 

Where an organisation sits on each of these axes is as much a matter of choice as a function of its history 
and organisational culture. But while some libraries are reinventing themselves, and refocusing energy 
on a range of service outputs, other seem almost to be trapped by their ovm histories. It is in the end a 
matter or choice. And for an organisation which wishes to shift where it sits on the axes representing 
values and focus, the axis representing purpose becomes critical. 

Like the typology employed by Cameron, the Values/focus/purpose matrix helps us see the relationship 
between various systems of performance measurement an devaluation. It emphasises the complex and 
multi-dimensional nature of organisational effectiveness, and therefore of the task of performance 
measurement. But the Values/focus/purpose matrix demonstrates even more clearly than the Cameron 
typology the element of choice and purpose that is fundamental to performance measurement. Every 
organisation can reposition itself on each of the three axes as it determines its relationship with its 
environment, its focus and its values. 

FACTORS AFFECTING THE USE OF SYSTEMS OF EVALUATION 
Do libraries make use of past research? 

Turning now to the fourth avenue by which we set out to address the question of whether performance 
measurement improves library effectiveness, i.e., by examining the factors that might improve or hinder 
the adoption of performance measurement in library and information service management, one question 
immediately arises. In all the literature that we have been discussing here there have been considerable 
insights into measurement and its value to libraries. Why has there not been greater uptake of this 
scholarship and research? Why is there so little agreement amongst the profession at large of the best 
approach to take to this problem, when library services are in so may other ways are becoming more and 
more standardised? 
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There has, in fact, been some uptake of researched-based management methodology within our 
profession in the past. From the first ALA manual (De Prospo, 1973) through to the last (Van House, 
1987) the methodologies have been well based on previous investigation and research, even if this was 
not entirely empirically based. Those libraries that have employed these or related methodologies report 
detectable increases in a range of output areas (Stephens, 1995) and some satisfaction with evaluation as 
an activity. But total commitment in the profession to the task seems to be still falling far short of what 
we might expect after such a lengthy gestation. In most institutions, it seems, measurement is still not 
feeding into the planning process; many libraries have little understanding of the measurement process, 
pay little heed to research, and few are able to document improved outcomes from their evaluation. 

Compared with other disciplines, it could be said, we do not make enough use of research to improve 
services or practice. A revealing comparison might be made with medicine. Despite political issues 
surrounding the delivery of medical care medicine is a highly professional discipline that uses its 
research and its literature to inform practice-there is a lot at stake, and the health professionals can't 
afford not to use the best information available. 

What is different about library service, and about information services generally? Do we see the same 
uptake of information from research? the same focus on improvement? the same awareness of the risks 
of inaction? If we go back to the 'peaks' in our landscape , and pick out some of the notable research and 
the breakthroughs of the past that could have changed professional practice, we have to ask to what 
extent have they changed anything? What impact has the work of Orr? of Kantor? of Altman? or Van 
House and Childers had? Well, obviously we do see changes, and we do see improvements. The 
academic library of today is more sophisticated, and offers more services than 25-30 years ago, when the 
first investigations to which this paper refers were conducted. Two notable changes have been in the area 
of reference services and Information Technology (one we can take credit for, the other possibly not). 

But are these the result of utilisation of research, a conscious attempt to use data and feedback to 
improve services? It seems that many of our improvements are ad hoc, rather than empirically based; 
they are reactive, i.e. responding to changes in the environment, or they originate with vendors driven by 
the need to get a competitive advantage, rather than being proactive, and led by the profession itself. 

Why? There are two possible reasons: 

1 . The incentives aren't strong enough - information management and services are rarely matters of 
life and death; they don't have to make a profit, satisfy shareholders, and until recently rarely had 
to prove their value. 

2. In attempting to measure library/information service performance we are dealing with some very 
imprecise outcomes/impacts - it is not always clear what is the 'best book' especially for a 
particular client, the 'best information’ or even the most up-to-date - no research will tell us this. 
The outcomes we talked of earlier, the social impacts, have not yet been defined adequately. Only 
in the business environment has there been any attempt to define the cost/benefit of the 
information service to the organisation. Our profession involves a complex area of human 
behaviour, we are uncertain of the benefits, and unlike medicine, we can't measure the value of our 
work - or the negative impact if it is done badly. 

In sum, as a profession, we have not embraced performance measurement in the decisive way that we 
have adopted technology. Clearer benefits might provide clearer incentives, and the combination of 
inadequate incentives, and fuzzy outcomes has led to a level of uncertainty and indecisiveness in action. 
But it is the first that interests me most for the time being, the question of incentives. 

CONCLUSIONS 

There would seem to be three critical factors influencing the positive outcome of performance measures 
in libraries. The three factors are inter-related, and form a useful framework for a final discussion of the 
issues. 

1. Measurement is a political activity 
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Performance measurement is a highly political activity, and must he seen as such, at the macro and the 
micro level. We must look outwards to social and political expectations made of our institutions and 
ensure that they meet the needs and expectations of our significant client or stakeholder groups; we must 
use our planning and goal-setting activities in a meaningful way, incorporating appropriate measures, to 
demonstrate our response to this external environment, and our willingness to align our aspirations to 
broader corporate goals. But we must also look within and seek to promote an organisational culture 
which acknowledges the political nature of measurement. This means using performance measurement 
to: 

• indicate the library or information service's alignment with broader organisational goals, 

• demonstrate the integration of information services with the key activities of the organisation, or 
of the community 

• support the library's position as the organisation's primary information manager and service 



Returning to the Values/focus/purpose matrix a library or information service wishing to define itself in 
this way will be careful to direct its energies and its performance measurement towards the outputs end 
of the values axis, choose an external orientation and develop a strong sense of purpose. 

2. The Multidimensional nature of performance measurement 

The application of Cameron's four models to existing modes of measurement and the 
Values/focus/purpose matrix have demonstrated that performance measurement is fundamentally 
multidimensional in nature. A library or information service that wishes to really understand how it is 
performing will examine both its environment and its constituencies, investigate the needs and 
expectations of its constituencies, examine its inputs and effectiveness in gaining resources, set goals 
which will allocate resources to respond to its various constituencies, measure efficiency and 
effectiveness in using resources, incorporate feedback into planning, revise goals in dialogue with 
various constituencies identified. You might describe this as the old systems model. It is far more than 
that. All four models outlined by Cameron must all be reflected in the evaluative procedures employed 
by the institution. It must recognise where it sits and consciously position itself on each one of the axes 
of the Values/focus/purpose matrix. 

3. Rewards and incentives 

Finally, we will not see informed and effective performance measurement in libraries until we can have 
got the right incentives in place . Again, these are both external and internal - they include the incentives 
and demands made by governments, local bodies, funding agencies, and parent organisations to provide 
evidence that the organisation is setting appropriate goals, meeting the expectations of a variety of 
stakeholders, and efficient in its conversion of inputs to outputs. And they include the incentives offered 
internally to reward good performance that is in line with organisation's objectives. In too many of our 
institutions poor performance is inadvertently rewarded, and good performance goes unacknowledged. 
An organisational culture is ideally attuned to the organisation's puiposes, and reflects the resolve of the 
organisation in whichever direction that takes it is. Again the organisation must consciously place itself 
as an organisation of strong resolve on the purpose axis of the Values/focus/purpose matrix. 

The question reput: a post-modern analysis 

We began this discussion by observing that the concept of 'library' and therefore the concept of 'library 
effectiveness' is a social construct which allows us to impose a discourse with which to define and 
discuss the concept. And it was suggested that the question of whether performance measurement 
improves library effectiveness is essentially unanswerable, because it is self-referential. But perhaps in 
this exploration of past and present approaches to measurement, and in the two multi-dimensional 
conceptual frameworks we used to define the problem, we can find an answer of sorts. Because this 
exploration has shown that since a library is a social construct, and performance measurement is a 
consequent social constmct, we are then free to both explore the definition of 'library' being imposed by 
any one system of measurement, and to ehose whieh definition of 'library' to employ. That is we ean 
adopt a system of measurement whieh best serves our definition and our purpose. The numerous 
dimensions of performanee measurement eneompass a range of methodologies and paradigms. Eaeh has 
its own perspective on what a library is about to bring to the task. Each has its own internal principles. 
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and imposes its own discourse. 

Libraries, and other organisations for that matter, in choosing between these various paradigms, are able 
to determine their placement on the three axes of the Values/focus/purpose matrix, are free to do so and 
must choose where and how to do so. There are no absolutes, so gurus to follow, no guarantees. Should 
they choose to focus their energies and their measurement on improved organisational effectiveness then 
that is likely be the outcome. 

We have known all this for the past decade. As Chuck McClure said in his 1 986 'Report from the 
trenches' what is needed are reliable methodologies, but even more so the professional leadership and 
organisational development, to make measurement an effective tool for libraries( McClure, 1986). A 
decade later we are still looking for new paradigms, testing new methodologies when they are already 
there. Performance measurement is an essential management tool, that may be implemented in a variety 
of ways. Each brings some desirable outcomes and comes with some disadvantages. Understanding the 
nature of each paradigm, and the underlying concept of the organisation which it implies, will help us 
make more effective choices from this range of methods. With this understanding, and with the 
leadership and organisational resolve to use measurement as a tool to increase organisational 
effectiveness, the possibilities are endless. 

The question may be self-referential. The answer I have outlined will, I hope, help you identify relevant 
and meaningful measures for your own library/information services. 

Notes 

^ A scond important characteristic of this research, again demonstrated by Van House and Childers, and Cullen and Calvert, 
is the potential to select with confidence a single measure from each dimension as a "surrogate" for other measures in the 
same dimension. This means that parsimonious measures to be selected which still covers all dimensions of library activity 
and evaluation (Cullen and Calvert, 1996). 

^ Of course many organisations may continue to be unresponsive to client needs whilst claiming to be client-centered. In 
such a case 'client-centered' remains an aspiration not an attribute. 
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